<html>
<head><meta charset="utf-8"><title>The short-term performance process review · t-compiler/performance · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/index.html">t-compiler/performance</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html">The short-term performance process review</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="226827839"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/226827839" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#226827839">(Feb 18 2021 at 15:32)</a>:</h4>
<p><span class="user-mention" data-user-id="116083">@pnkfelix</span> : I wrote a document on the short-term performance tracking process. This includes the current status quo as well as my idea of an ideal process which we can use to spring board discussion and reach a consensus on an ideal process. This is only for short-term performance tracking (i.e., ensuring that PRs introduce as few regressions as possible and ensuring that they are either fixed or labeled as wont-fix). How we do longer term performance tracking (i.e., keeping the compiler team and wider community informed about trends in performance and making sure that trend is toward improving performance) is left to another exercise. <a href="https://hackmd.io/@ryanlevick/Bk1Hidcbu">https://hackmd.io/@ryanlevick/Bk1Hidcbu</a></p>



<a name="226882873"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/226882873" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#226882873">(Feb 18 2021 at 21:21)</a>:</h4>
<p>thanks <span class="user-mention" data-user-id="224872">@rylev</span> !</p>



<a name="226939475"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/226939475" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#226939475">(Feb 19 2021 at 09:32)</a>:</h4>
<p>Next steps would be to give the document a read and get to a place where everyone agrees on the description of the status quo and agrees enough with this proposed direction that we can bring it to a wider audience (probably t-compiler?).</p>



<a name="227569471"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227569471" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227569471">(Feb 24 2021 at 09:18)</a>:</h4>
<p><span class="user-mention" data-user-id="116083">@pnkfelix</span> Do you have time to discuss this ^^</p>



<a name="227587793"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227587793" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227587793">(Feb 24 2021 at 12:03)</a>:</h4>
<p>I just read it - minor feedback is that I recommend filing new issues, merged PRs just don't work for tracking anything I think</p>



<a name="227588091"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227588091" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227588091">(Feb 24 2021 at 12:06)</a>:</h4>
<p>I also wanted to flag that historically at least, there's not been a huge amount of success with nagging people to do work - I suspect that the performance team itself will need to make PRs aiming to fix performance or revert regressions</p>



<a name="227589528"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227589528" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227589528">(Feb 24 2021 at 12:21)</a>:</h4>
<p>I agree that nagging hasn't been successful so far, but I do think that would change in a world where the nagging is followed up on. I do agree though that the perf team itself may need to follow up on issues, but I don't want to get to a situation where people can just rely on the perf team fixing any performance regressions they may introduce.</p>



<a name="227590699"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227590699" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227590699">(Feb 24 2021 at 12:33)</a>:</h4>
<p>Agreed, I definitely don't want that, and I think one strategy is to have a default of reverting and asking to reapply regression free (or otherwise mitigating), though this may not be trivial in practice to get right.</p>



<a name="227591784"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227591784" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227591784">(Feb 24 2021 at 12:42)</a>:</h4>
<p>This depends on how seriously we want to take regressions. if performance decreases by 2% on one benchmark, I believe an investigation is warranted but a revert might be overkill.</p>



<a name="227595929"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227595929" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227595929">(Feb 24 2021 at 13:17)</a>:</h4>
<p>I don't disagree but I think we've historically struggled with getting investigation without an obvious reason to do it. Maybe I'm too pessimistic, but I'd rather revert or not land regressing PRs without that investigation done up front. We could also get stricter over time as tooling to do that investigation improves, though.</p>



<a name="227599757"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227599757" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227599757">(Feb 24 2021 at 13:45)</a>:</h4>
<p>Yea let's start with a more "optimistic" process and we can crack down if we need to.</p>



<a name="227626395"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227626395" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227626395">(Feb 24 2021 at 16:23)</a>:</h4>
<p>Would it help if we changed the format of the weekly report?</p>



<a name="227626470"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227626470" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227626470">(Feb 24 2021 at 16:23)</a>:</h4>
<p>to focus on just “high level, performance went up or went down overall (and by how much)”, and then “here are the nags"</p>



<a name="227626623"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227626623" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227626623">(Feb 24 2021 at 16:24)</a>:</h4>
<p>that is, if the nags are the real point of the synchronous report with the team, then maybe they should get more weight. Arguably that’s hard without the rest of the context provided by the report as a whole.</p>



<a name="227626724"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227626724" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227626724">(Feb 24 2021 at 16:24)</a>:</h4>
<p>but if the intent is that we need someone to <em>own</em> the action item of pushing on the nag, we can adapt things to make that happen.</p>



<a name="227627031"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227627031" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227627031">(Feb 24 2021 at 16:26)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116122">simulacrum</span> <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/The.20short-term.20performance.20process.20review/near/227590699">said</a>:</p>
<blockquote>
<p>Agreed, I definitely don't want that, and I think one strategy is to have a default of reverting and asking to reapply regression free (or otherwise mitigating), though this may not be trivial in practice to get right.</p>
</blockquote>
<p>I expect this strategy will be difficult until we are very proactive. I.e. the weekly cadence may not be quick enough, in terms of the pain it will introduce to others who are rebasing and then have to undo their rebasing.</p>



<a name="227627217"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227627217" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227627217">(Feb 24 2021 at 16:27)</a>:</h4>
<p><span class="user-mention silent" data-user-id="224872">rylev</span> <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/The.20short-term.20performance.20process.20review/near/227569471">said</a>:</p>
<blockquote>
<p><span class="user-mention silent" data-user-id="116083">pnkfelix</span> Do you have time to discuss this ^^</p>
</blockquote>
<p>Would this be worthy of a Friday steering meeting? I’m reading it now with that in mind, but possibly we are better off with a more focused discussion here.</p>



<a name="227627966"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227627966" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227627966">(Feb 24 2021 at 16:31)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116083">pnkfelix</span> <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/The.20short-term.20performance.20process.20review/near/227626470">said</a>:</p>
<blockquote>
<p>to focus on just “high level, performance went up or went down overall (and by how much)”, and then “here are the nags"</p>
</blockquote>
<p>I think keeping general information about the trend for the week probably wouldn't hurt, though I haven't see a case yet where that really comes in handy. I think we won't know the final form the triage report should take until we do more to review the long term performance tracking process.</p>



<a name="227628247"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227628247" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227628247">(Feb 24 2021 at 16:32)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116083">pnkfelix</span> <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/The.20short-term.20performance.20process.20review/near/227627217">said</a>:</p>
<blockquote>
<p>Would this be worthy of a Friday steering meeting? I’m reading it now with that in mind, but possibly we are better off with a more focused discussion here.</p>
</blockquote>
<p>I'll leave that decision up to you and how controversial you think the proposal is/will be. I'm more than happy to discuss it in a meeting though</p>



<a name="227637179"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227637179" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227637179">(Feb 24 2021 at 17:21)</a>:</h4>
<p>Rather than talking about reverting or investigating, isn't the goal to raise the issue earlier in the process, i.e.</p>
<blockquote>
<p>When the performance run is finished, the perf bot uses heuristics to determine whether this run has found a possible performance regression. If it has, the bot automatically labels the pull request as such (actual label name to be bikeshed) and encourages the reviewer and author to address the regression.</p>
</blockquote>



<a name="227639520"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227639520" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227639520">(Feb 24 2021 at 17:34)</a>:</h4>
<p>Yes ideally we would never reach the point where a merged PR shows a performance regression. This is just the contingency when that does happen.</p>



<a name="227653397"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227653397" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227653397">(Feb 24 2021 at 19:02)</a>:</h4>
<p><span class="user-mention silent" data-user-id="224872">rylev</span> <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/The.20short-term.20performance.20process.20review/near/227639520">said</a>:</p>
<blockquote>
<p>Yes ideally we would never reach the point where a merged PR shows a performance regression. This is just the contingency when that does happen.</p>
</blockquote>
<p>I would just clarify -- <em>unexpected</em> regression :)</p>



<a name="227765278"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227765278" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227765278">(Feb 25 2021 at 13:20)</a>:</h4>
<p>Ok I've updated the document to address the feedback and added more detail to the "weekly report". It's shaping up to be somewhat similar to the current triage report just without any human editing needed and more focused on regressions (though improvements are still noted).</p>



<a name="227765355"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227765355" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227765355">(Feb 25 2021 at 13:21)</a>:</h4>
<p><span class="user-mention" data-user-id="116083">@pnkfelix</span> perhaps this would be interested to announce at the compiler triage meeting and ask people if they believe an MCP is warranted.</p>



<a name="227765405"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227765405" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227765405">(Feb 25 2021 at 13:21)</a>:</h4>
<p>I'd like to get more feedback and start working with folks on implementing this.</p>



<a name="227779131"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227779131" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227779131">(Feb 25 2021 at 14:54)</a>:</h4>
<p>sure, I think announcing it makes sense.</p>



<a name="227779193"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/227779193" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#227779193">(Feb 25 2021 at 14:55)</a>:</h4>
<p><span class="user-mention" data-user-id="224872">@rylev</span> are you going to attend the meeting? You can just make an announcement during the initial announcement period; its meant to be open to everyone</p>



<a name="228427366"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/228427366" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#228427366">(Mar 02 2021 at 15:21)</a>:</h4>
<p><span class="user-mention" data-user-id="116083">@pnkfelix</span> I haven't gotten any comments on this. Is it worth making it an MCP?</p>



<a name="228431000"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/228431000" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#228431000">(Mar 02 2021 at 15:42)</a>:</h4>
<p>Are you thinking the MCP would be the section labelled "A Possible Ideal" ?</p>



<a name="228431148"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/228431148" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#228431148">(Mar 02 2021 at 15:43)</a>:</h4>
<p>Yes. I personally would be happy supporting that idea. <span class="user-mention" data-user-id="116122">@simulacrum</span> are you happy with the proposal or would you like to see some changes?</p>



<a name="228431303"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/228431303" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#228431303">(Mar 02 2021 at 15:44)</a>:</h4>
<p>It might be good if you highlight the bullets, or the subparts of bullets, that represent changes over the status quo</p>



<a name="228431421"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/228431421" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#228431421">(Mar 02 2021 at 15:44)</a>:</h4>
<p>(e.g. add "(new)" at the start of the second bullet, and in front of every other entirely new bullet.)</p>



<a name="228431689"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/228431689" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#228431689">(Mar 02 2021 at 15:46)</a>:</h4>
<p>I can take a look in a bit.</p>



<a name="228436819"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/228436819" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#228436819">(Mar 02 2021 at 16:14)</a>:</h4>
<p>Seems reasonable to me, but it'll definitely need to get implemented by "not me" :)</p>



<a name="228437075"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/228437075" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#228437075">(Mar 02 2021 at 16:15)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116083">pnkfelix</span> <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/The.20short-term.20performance.20process.20review/near/228431421">said</a>:</p>
<blockquote>
<p>(e.g. add "(new)" at the start of the second bullet, and in front of every other entirely new bullet.)</p>
</blockquote>
<p>I can do this though almost every bullet might be marked as such <span aria-label="blush" class="emoji emoji-1f60a" role="img" title="blush">:blush:</span></p>



<a name="228437147"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/228437147" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#228437147">(Mar 02 2021 at 16:16)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116122">simulacrum</span> <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/The.20short-term.20performance.20process.20review/near/228436819">said</a>:</p>
<blockquote>
<p>Seems reasonable to me, but it'll definitely need to get implemented by "not me" :)</p>
</blockquote>
<p>I'd actually like to implement this myself. I want to learn more about the bots.</p>



<a name="228488352"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/228488352" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Tyson Nottingham <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#228488352">(Mar 02 2021 at 21:23)</a>:</h4>
<p>The proposal seems good!</p>
<p>My only concern is that we won't have the infrastructure to support as many perf runs as we'd like. If it turns out to be too much of a load on the infrastructure, we might look at tuning the heuristic to suggest full perf runs less often, and integrate lightweight smoke tests, as I talked about <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/lets.20keep.20talking.20about.20perf.2Erlo/near/225359280">here</a>. And / or convince someone to sponsor some hardware. :)</p>



<a name="228499883"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/228499883" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Tyson Nottingham <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#228499883">(Mar 02 2021 at 22:39)</a>:</h4>
<p>Also, some thoughts on determining whether or not there was a regression:</p>
<p>The heuristic should do some kind of analysis which takes into account the history of the particular benchmark. For example, for max-rss, a 5% increase can be within noise for one benchmark (e.g. keccak-opt), but be an obvious regression for another benchmark (e.g. clap-rs-opt).</p>
<p>For triage, we have more information at our disposal. We can often see that a change that was within noise actually turned out to be a real regression after all. See <a href="https://github.com/rust-lang/rustc-perf/issues/138">https://github.com/rust-lang/rustc-perf/issues/138</a>.</p>



<a name="230315677"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/230315677" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#230315677">(Mar 15 2021 at 09:29)</a>:</h4>
<p><span class="user-mention" data-user-id="116083">@pnkfelix</span> thanks for posting this in more places. Do you have a brief moment to describe next steps? I'd like to see if we can at least get this in a tracking issue so I can work on some of these issues and recruit others to help.</p>



<a name="230359786"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/230359786" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#230359786">(Mar 15 2021 at 15:05)</a>:</h4>
<p>hi <span class="user-mention" data-user-id="224872">@rylev</span> . Do you just want to talk asynchronously in here?</p>



<a name="230359943"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/The%20short-term%20performance%20process%20review/near/230359943" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> rylev <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/The.20short-term.20performance.20process.20review.html#230359943">(Mar 15 2021 at 15:06)</a>:</h4>
<p>Yes (In a meeting right now so I'll be slow to respond right now)</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>